Prognostic models for COVID-19 needed updating to warrant transportability over time and space.

BACKGROUND: Supporting decisions for patients who present to the emergency department (ED) with COVID-19 requires accurate prognostication. We aimed to evaluate prognostic models for predicting outcomes in hospitalized patients with COVID-19, in different locations and across time. METHODS: We included patients who presented to the ED with suspected COVID-19 and were admitted to 12 hospitals in the New York City (NYC) area and 4 large Dutch hospitals. We used second-wave patients who presented between September and December 2020 (2137 and 3252 in NYC and the Netherlands, respectively) to evaluate models that were developed on first-wave patients who presented between March and August 2020 (12,163 and 5831). We evaluated two prognostic models for in-hospital death: The Northwell COVID-19 Survival (NOCOS) model was developed on NYC data and the COVID Outcome Prediction in the Emergency Department (COPE) model was developed on Dutch data. These models were validated on subsequent second-wave data at the same site (temporal validation) and at the other site (geographic validation). We assessed model performance by the Area Under the receiver operating characteristic Curve (AUC), by the E-statistic, and by net benefit. RESULTS: Twenty-eight-day mortality was considerably higher in the NYC first-wave data (21.0%), compared to the second-wave (10.1%) and the Dutch data (first wave 10.8%; second wave 10.0%). COPE discriminated well at temporal validation (AUC 0.82), with excellent calibration (E-statistic 0.8%). At geographic validation, discrimination was satisfactory (AUC 0.78), but with moderate over-prediction of mortality risk, particularly in higher-risk patients (E-statistic 2.9%). While discrimination was adequate when NOCOS was tested on second-wave NYC data (AUC 0.77), NOCOS systematically overestimated the mortality risk (E-statistic 5.1%). Discrimination in the Dutch data was good (AUC 0.81), but with over-prediction of risk, particularly in lower-risk patients (E-statistic 4.0%). Recalibration of COPE and NOCOS led to limited net benefit improvement in Dutch data, but to substantial net benefit improvement in NYC data. CONCLUSIONS: NOCOS performed moderately worse than COPE, probably reflecting unique aspects of the early pandemic in NYC. Frequent updating of prognostic models is likely to be required for transportability over time and space during a dynamic pandemic.

Development and validation of self-monitoring auto-updating prognostic models of survival for hospitalized COVID-19 patients.

Clinical prognostic models can assist patient care decisions. However, their performance can drift over time and location, necessitating model monitoring and updating. Despite rapid and significant changes during the pandemic, prognostic models for COVID-19 patients do not currently account for these drifts. We develop a framework for continuously monitoring and updating prognostic models and apply it to predict 28-day survival in COVID-19 patients. We use demographic, laboratory, and clinical data from electronic health records of 34912 hospitalized COVID-19 patients from March 2020 until May 2022 and compare three modeling methods. Model calibration performance drift is immediately detected with minor fluctuations in discrimination. The overall calibration on the prospective validation cohort is significantly improved when comparing the dynamically updated models against their static counterparts. Our findings suggest that, using this framework, models remain accurate and well-calibrated across various waves, variants, race and sex and yield positive net-benefits.

High Frequency of COVID-19 Vaccine Hesitancy among Canadians Immunized for Influenza: A Cross-Sectional Survey.

We assessed the frequency and correlates of COVID-19 vaccine hesitancy before Canada's vaccine rollout. A cross-sectional vaccine hesitancy survey was completed by consecutive patients/family members/staff who received the influenza vaccine at McGill University affiliated hospitals. Based on the self-reported likelihood of receiving a future vaccine (scale 0-10), the following three groups were defined: non-hesitant (score 10), mildly hesitant (7.1-9.9), and significantly hesitant (0-7). Factors associated with vaccine hesitancy were assessed with multivariate logistic regression analyses and binomial logistic regression machine learning modelling. The survey was completed by 1793 people. Thirty-seven percent of participants (n = 669) were hesitant (mildly: 315 (17.6%); significantly: 354 (19.7%)). Lower education levels, opposition and uncertainty about vaccines being mandatory, feelings of not receiving enough information about COVID-19 prevention, perceived social pressure to get a future vaccine, vaccine safety concerns, uncertainty regarding the vaccine risk-benefit ratio, and distrust towards pharmaceutical companies were factors associated with vaccine hesitancy. Vaccine safety concerns and opposition to mandatory vaccinations were the strongest correlates of vaccine hesitancy in both the logistic regressions and the machine learning model. In conclusion, in this study, over a third of people immunized for influenza before the COVID-19 vaccine rollout expressed some degree of vaccine hesitancy. Effectively addressing COVID-19 vaccine safety concerns may enhance vaccine uptake.

Development and Validation of a Survival Calculator for Hospitalized Patients with COVID-19.

BACKGROUND: Chinese studies reported predictors of severe disease and mortality associated with coronavirus disease 2019 (COVID-19). A generalizable and simple survival calculator based on data from US patients hospitalized with COVID-19 has not yet been introduced. OBJECTIVE: Develop and validate a clinical tool to predict 7-day survival in patients hospitalized with COVID-19. DESIGN: Retrospective and prospective cohort study. SETTING: Thirteen acute care hospitals in the New York City area. PARTICIPANTS: Adult patients hospitalized with a confirmed diagnosis of COVID-19. The development and internal validation cohort included patients hospitalized between March 1 and May 6, 2020. The external validation cohort included patients hospitalized between March 1 and May 5, 2020. MEASUREMENTS: Demographic, laboratory, clinical, and outcome data were extracted from the electronic health record. Optimal predictors and performance were identified using least absolute shrinkage and selection operator (LASSO) regression with receiver operating characteristic curves and measurements of area under the curve (AUC). RESULTS: The development and internal validation cohort included 11 095 patients with a median age of 65 years [interquartile range (IQR) 54-77]. Overall 7-day survival was 89%. Serum blood urea nitrogen, age, absolute neutrophil count, red cell distribution width, oxygen saturation, and serum sodium were identified as the 6 optimal of 42 possible predictors of survival. These factors constitute the NOCOS (Northwell COVID-19 Survival) Calculator. Performance in the internal validation, prospective validation, and external validation were marked by AUCs of 0.86, 0.82, and 0.82, respectively. LIMITATIONS: All participants were hospitalized within the New York City area. CONCLUSIONS: The NOCOS Calculator uses 6 factors routinely available at hospital admission to predict 7-day survival for patients hospitalized with COVID-19. The calculator is publicly available at https://feinstein.northwell.edu/NOCOS.

Machine learning to assist clinical decision-making during the COVID-19 pandemic.

BACKGROUND: The number of cases from the coronavirus disease 2019 (COVID-19) global pandemic has overwhelmed existing medical facilities and forced clinicians, patients, and families to make pivotal decisions with limited time and information. MAIN BODY: While machine learning (ML) methods have been previously used to augment clinical decisions, there is now a demand for "Emergency ML." Throughout the patient care pathway, there are opportunities for ML-supported decisions based on collected vitals, laboratory results, medication orders, and comorbidities. With rapidly growing datasets, there also remain important considerations when developing and validating ML models. CONCLUSION: This perspective highlights the utility of evidence-based prediction tools in a number of clinical settings, and how similar models can be deployed during the COVID-19 pandemic to guide hospital frontlines and healthcare administrators to make informed decisions about patient care and managing hospital volume.